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Abstract 

In a world in which online interactions are becoming the norm, an 
understanding of how three fundamental aspects of online learning 
(teaching presence, social presence and cognitive presence) interact is 
important. This paper will look at how these three presences interact with 
each other in an online forum. More specifically it will describe the 
effects of instructional design on learners’ levels of critical thinking and 
social presence. The research involved taking 900 learner posts from 
differing experimental conditions and analyzing those posts for social 
presence and critical thinking. The experimental conditions varied in 
three different ways in regards to the level of instructor control over the 
learning environment. The first learning environment had a low level of 
instructor control while the second and third had progressively high 
levels of instructor control over the learners’ contributions to the forum. 
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The results showed that increasing the amount of control an instructor has 
over a learning environment increases the amount of cognitive presence 
but decreases the amount of social presence within the learners’ posts. In 
general, these results are important because instructors must be aware of 
how their behavior may affect how learners interact (and therefore leam) 
online. More specifically, many instructors are interested in the types of 
discourse their learners create. Therefore, the ways instructors can 
manipulate learner discourse is of great importance. 

Keywords: Asynchronous forum, critical thinking, instructor control, 
social presence, task design 


INTRODUCTION 

Many universities are moving towards offering courses with 
significant online components. These courses often combine some type 
of live brick and motor instruction with an asynchronous learning 
platform (Holenko & Hoic-Bozic, 2008; Precel, Eshet-Alkalai, & 
Alberton, 2009). Asynchronous discussion boards are the most 
commonly used tool to generate learner-centered discussions in higher 
education settings (Johnson, 2007). The technology allows university 
students to ask questions, discuss issues, and observe how their peers are 
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interacting with the content of any particular course. Their type of 
interaction is also important, with social interaction being shown to 
support learning (Kozan & Richardson, 2014). Knowledge of how 
instructors should use asynchronous online forums is important, as 
implementation has been haphazard in some cases (US Department of 
Education, 2010). This new form of interaction gives instructors new 
options and challenges in how to construct learning events for their 
students. These new challenges of how to construct and administer 
learning environments requires an understanding of how instructors can 
design and administer instruction to the growing number of learners who 
participate on asynchronous learning networks. 

The effects of instruction on learner behavior is an important 
component of understanding online learning and certain types of 
instruction have been shown to develop learners’ levels of critical 
thinking and increase learner confidence (Heijltjes, van Gog, Leppink & 
Pass, 2015). Furthermore, regardless of the learner orientated nature of 
most educational discussion forums, the role of the teacher remains of 
great importance, as the teacher still has responsibility to: design the 
learners’ educational experience, facilitate the learning experience, and to 
provide scaffolding in the fonn of direct instruction as needed (Anderson, 
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Rourke, Garrison, & Archer, 2001). Furthermore, studies have 
established the importance and effectiveness of instructor behavior when 
students interact online (Andresen, 2009; Shea, Chun, & Pickett, 2006) 
and the effect of teaching presence on critical thinking in particular 
(Prasad, 2009). Individual learning without the aid of formal instruction 
can be effective online, however some kind of guidance (teaching 
presence) is required (Anderson et al., 2001). 

The ability of e-learning, particularly asynchronous online 
forums, to develop both private reflection and higher order discourse 
through collaborative learning shows that e-leaming has the 
characteristics to develop independent critical thinkers (Nui, Behar- 
Horenson & Garven, 2013). Effective collaboration requires at least some 
level of discourse, and the more in depth the discourse the more in depth 
the collaboration. 

This study investigates the effects of instructional design and 
instructor posting on student discussion in online threaded asynchronous 
forums. To gather a rich understanding of learner behavior in relation to 
instructor behavior, direct analysis of student discussions is needed to 
develop a comprehensive understanding of how instructor behavior can 
effect learner discussion. The two tools used to understand student 
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discussions are the concepts of social presence and critical thinking. This 
paper will describe the effects that varying types of instructor behavior 
have on the levels of social presence and critical thinking within learner 
discussions. 

CMC in Education 

While effective learning communities exist offline, the power of 
e-leaming is its ability to easily bring together diverse communities of 
learners. E-learning in general and the Internet specifically is a powerful 
tool to reach deeper layers of understanding, raise participation, and 
promote motivation. This is because of its potential social diversity 
(learners can interact with nearly anyone) and content depth and diversity 
(learners can find information about nearly anything). 

Asynchronous online discussion forums have been used to 
facilitate online learning and, according to Harman & Koohang (2005), 
are one of the primary tools of CMC in education. The use of online 
discussion forums can be used in three fundamentally separate ways: as 
the only method of interaction for learners, as a method along with other 
online computer based interactions, or as a method used in conjunction 
with offline interactions. In all three cases, online discussion forums are 
an effective, simple way to facilitate collaboration and interaction 
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(Harman & Koohang, 2005; Reid, Katz & Jacobsen, 2006). Students 
have been shown to prefer asynchronous offline interactions when 
compared to online or offline synchronous. This is because the slow pace 
allows the students to respond in their own time and in their own way 
(Callan, 2006; O’Neill et ah, 2006; Wang & Woo, 2007). Furthermore 
Hara, Bonk, and Angeli (2002) claimed that, asynchronous online 
responses were of a higher quality (in that they were longer, more in- 
depth, and thoughtful) than face-to-face responses of similar problem 
types. Nearly all asynchronous online forums are in written form. Writing 
in and of itself encourages learners to reflect not only on whatever 
environment, artifact or actor they are interacting with, but also on the 
learner’s own thoughts and beliefs (Hiemstra, 2002; Spalding & Wilson, 
2002 ). 

Learner engagement and feelings of belonging are core issues 
when designing any learning community and learning communities 
online are no exception (Vesely, Bloom, & Sherlock, 2007). Simply 
speaking, a teacher will be unable to deliver effective instruction and 
learners will be unable to meet learning goals if the learning environment 
doesn’t meet learners’ basic social needs (Kozan & Richardson, 2014). 
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The Community of Inquiry (Col) is one of the most commonly 
used and validated models in online education research (Lee, 2013). 
According to Lipman, Sharp, and Oscanyan (1980), a community of 
inquiry has 3 core prerequisites: 1) Readiness to reason; 2) Mutual 
Respect (among learners and between the learners and teachers); 3) An 
absence of indoctrination. These concepts have been applied to the e- 
leaming environment most notably in a paper by Garrison, Anderson and 
Archer (2000), which laid out a model for understanding how the pieces 
of the community of inquiry worked together. Their paper provides a 
conceptual tool for understanding computer-mediated communication 
and a model of a community of inquiry. Their model, shown in figure 1 is 
made up of three main parts: social presence, cognitive presence and 
teaching presence. 


The Journal of Educators Online-JEO January 2016 ISSN 1547-500X 

Vol 13 Number 1 

116 


Community of Inquiry 



Figure 1. Elements of an educational experience. (From Garrison, 
Anderson and Archer, 2000, p 88.) 

The element in the model that is core to the traditional view of 
“learning” is that of cognitive presence, which Garrison et al. define as, 

the extent to which the participants “.are able to construct meaning 

through sustained communication” (Garrison et al., 2000, p. 89). The 
second element in the model is social presence. The authors define social 
presence as when learners are able to show their personal characteristics 
to the community of inquiry. The nature of social presence is what 
divides a Col from normal learning. Garrison et al. state that, “it affects 
the quality of the message; the tone is questioning but engaging, 
expressive but responsive, skeptical but respectful, and challenging but 
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supportive” (pg. 96). The third element of the model, teaching presence, 
consists of three parts: design, facilitation, and instruction. According to 
Garrison et al. (2000) teaching presence is essentially a method of 
balancing the social and cognitive presences and moving them towards 
educational objectives. This may require a “teacher”, but teaching 
presence is not only encapsulated in that individual. Within the 
framework of the Col, the element that can be most easily manipulated 
by those seeking to improve education is teaching presence. According to 
Garrison and Anderson (2003), the role of teaching presence in the 
community of inquiry includes design and organization, facilitating 
discourse and direct instruction. More recent research by Lee (2013), has 
found that there are strong relationships between the constituent parts of 
the Col, and that each part has a role to play in an effective learning 
environment. 

Teaching Presence 

The student’s view of the teacher and how the teacher behaves is 
of great importance. In Midgley, Feldlaufer and Eccles (1989), they 
showed that the relationship that a student had with his or her teacher was 
highly correlated with that student’s feeling about the subject and 
achievement in the subject. They found that when students felt high 
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levels of teacher support, their attitudes towards the subject and results in 
the subject dramatically improved. Pianta (1999) found that, while 
external influences and individual characteristics did affect 
student/teacher information sharing, the defining feature of the 
relationship was the process by which the student and teacher engaged 
with each other. This has been shown more recently in research that has 
demonstrated the instructors influence on many aspects of learner 
behavior (Heijltjes, van Gog, Leppink & Pass, 2015; Jarvela & Hadwin, 
2013; Kim, 2015). In Raider-Roth’s (2005) detailed examination of 6 th 
grade learners, she found that students engagement with and ability to 
understand classroom material was based on their relationships with the 
teachers and their peers. More specifically, there was a complex but clear 
relationship between the teacher student relationship and affective 
disposition towards education and knowledge. 

Anderson et al. (2001) have come up with three categories that 
directly define the role of teaching presence in a community of inquiry: 
instructional design and organization, facilitating discourse, and direct 
instruction. Instructional design and organization involves 5 indicators: 
setting curriculum, designing methods, establishing time parameters, 
utilizing the medium effectively and establishing netiquette. They note 
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that students need a clear idea of the overall plan of the course so as to 
maintain focus and direction. While Anderson et al.’s (2001) model is 
useful in describing differing instructional design and organization, their 
conceptualization is operationalized as instructor utterances. While it is 
sometimes the case that the parameters of design and instruction will be 
delivered by the instructor in the online learning environment, the 
information may be delivered in a syllabus package or email in the case 
of fully online classes, or offline during in-class instruction in the case of 
blended learning environments (Arbuagh & Hwang, 2006). 

Other methods of understanding learning environments have been 
developed, most notably Reeves and Reeves (1997) who develop 10 
dimensions that can be used to understand pedagogic differences in web- 
based instructional environments. These 10 dimensions are: (1) 
pedagogical philosophy, (2) learning theory, (3) goal orientation, (4) task 
orientation, (5) source of motivation, (6) teacher role, (7) metacognitive 
support, (8) collaborative learning, (9) cultural sensitivity, and (10) 
structural flexibility. Each of these dimensions can vary along a 
continuum, and a particular learning environment can be pedagogically 
described by placing them at each point on the continuum for each of the 
10 dimensions. Reeves and Reeves (1997) use this model to describe two 
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differing learning environments from each other. This model allows a 
more precise description of learning environments than Anderson et al.’s 
(2001) model by not only having more dimensions, but by giving 
guidelines as to how those dimensions can vary. 

A modified version of Reeves and Reeves’ (1997) model was 
taken up by Siragusa, Dixon and Dixon (2007). Their mode, the 
Instructional Design for Online Learning model (IDOL), uses three 
differing categories: analysis, strategy and evaluation, with these three 
categories being developed into 24 varying dimensions of design and 
organization. There are 11 dimensions for analysis: underlying 
pedagogical philosophy, instructional design analysis, content provided, 
student motivation, unit formation / delivery mode, lecturers role / 
availability, lecturer’s perception of importance, lecturer’s online 
abilities, lecturers online support / training, lecturers decision making 
input, and lecturers development activities. There are a further 11 
dimensions within the category of strategy which are: structure and 
organization, development of learning strategies, content guiding learning 
strategies, accoimnodation of learning styles, study flexibility - when, 
where and pace, web based design principles, interaction, collaborative 
learning, automated online learning activities, internet based information, 
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and online learning management. Within the final category of evaluation 
there are two categories, which are feedback and online learning 
evaluation. Siragusa et al. (2007) describe two differing learning 
environment among the 24 dimensions, this gives a clear framework for 
our understanding the nature of these online communities. This paper will 
use a modified version of the IDOL model for describing the three 
differing learning environments used in this study, which will be 
described in greater detail in the methods section of this paper. 

The Effects of Teaching Presence on Social Presence 

Conceptions of social presence began with Mehrabian (1969) and 
his idea of immediacy. Immediacy is those actions by individuals, which 
bring them together and/or increase the interactions between individuals. 
Mehrabian was focused on non-discourse related social behaviors such as 
voice tone, volume and body language. McCroskey and Wheeless (1977) 
introduced the concept of affinity, defined as an individual’s positive 
attitude towards another individual. Affinity would, in McCroskey and 
Wheeless’ view, increase levels of communication between members of a 
community. Mehrabian (1969) along with McCroskey and Wheeless 
(1976) hypothesized that a lack of physical closeness or nonverbal 
behaviors would be detrimental to individual-to-individual 
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communication. This brings about a problem when trying to develop 
most kinds of asynchronous communication mediums online, as they lack 
any kind of nonverbal social cues. 

Gunawardena (1995) is based around a set of 17 Likert scale 
items that measured the degree of social presence learners felt in CMC 
learning environments. These items asked learners to score where on the 
scale they felt the learning environment was. For example, student 
perceptions of how active or passive or stimulating or dull they found the 
learning environment created an overall measure of the level to which a 
learning environment contained social presence. Gunawardena’s (1995) 
scale was further developed (Gunawardena & Zittle, 1997) with 14 items 
that asked participants their level of agreement with statements made 
about the learning environment. In 2002, Tu developed The Social 
Presence and Privacy Questionnaire (SPPQ). This survey was initially 
developed by 5 content experts. Then, in an experiment with 310 learners 
to test construct validity, factor analysis showed that five features stood 
out as being important. These features, which were social context, online 
communication, interactivity, system privacy, and feelings of privacy 
made up the core of measuring social presence. In further work, Tu and 
Mclssac (2002) found that more variables may impact social presence 
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and that the dimensions of social presence may need to be weighted for 
more accurate measurement. Rourke, Anderson, Garrison and Archer. 
(1999) took an important step in the measurement of social presence by 
looking at learner discourse directly as opposed to using a survey. This 
measurement of social presence was tested by analyzing student 
discourse in the form of transcripts from an asynchronous online learning 
class. This measurement tool is what is being used in this research and 
the full scheme can be seen in appendix 2. More recent research has 
shown that social presence is important because it can potentially 
improve levels of cognitive presence (Kozan & Richardson, 2014; Lee, 
2013). 

Jarvela and Had win (2013) claimed that for learners to interact 
and collaborate online, support should not only target critical thinking, 
but also develop learners’ motivation, social development and emotional 
development. Mazzolini and Maddison (2002) investigated the different 
ways instructors guide their learners online. They showed that 
instructors’ styles of interaction affected how learners felt and the degree 
that they participated online. Mazzolini and Maddison found that 
frequent posting by instructors led to discussions being cut short and did 
not lead to more posts by learners in the discussions. Learners who 
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experienced more facilitation from their instructors did rate their 
instructors higher and judged that the instructors were more enthusiastic 
than those who interacted less online. In their quantitative and qualitative 
investigation of social presence Tu and Mclsaac (2002) found that the 
three dimensions of social presence (social context, online 
communication and interactivity) could be enhanced by certain types of 
instructor behaviors. They recommend that instructors engage learners in 
social tasks and take steps to remove layers of formality between 
themselves and learners in order to promote social engagement. Swan 
and Shih (2005) demonstrated that instructor presence was important in 
developing the levels of social presence students felt in online courses. 
They found that topics which were more focused around personal issues 
induced higher levels of social presence. They also found that there were 
meaningful differences in levels of social presence when learners’ 
instructors behaved in different ways. Students with higher levels of 
social presence reported that their instructors had a more “personal tone” 
in their online interaction and that those instructors spent time developing 
a sense of community. Students who felt that their instructor didn’t 
“facilitate” much, often felt passive and bored when trying to relate with 
the class content. In further research by Shea, Chun and Pickett (2006) 
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into the effects of teaching presence on sense of community students’ 
perception of teaching presence was correlated with a survey of online 
community along with other demographic information. Their results 
showed strong relationships between teaching presence and learners’ 
sense of community. They propose two categories: design and 
organization and directed facilitation. They found that many of the 
features of direct instruction did not correlate with sense of community. 

In Shea, Fredrickson, Pickett and Pelz (2003), they investigated the 
components of teaching presence students are most likely to perceive. 
They used a survey of learners and found that certain features of teacher 
behavior were more highly correlated with student satisfaction and 
reported learning. In regards to instructional design and organization, 

Shea et al. (2003) found that there was a strong correlation between 
students perception of design and organization with satisfaction (r = .635) 
and reported learning (r = .588). Interestingly, Shea et al. found that 
facilitating discourse had the same relationship with satisfaction (r = .64) 
and learning (r = .58). The effects of direct instruction on satisfaction (r = 
.64) were the same as the relationship between organization and design, 
and facilitating discourse. However, direct instruction had a slightly 
higher effect on reported learning (r = .61) than facilitating discourse or 
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design and instruction had. While this research was indirect, as it used 
student perceptions to measure both the independent variables (teaching 
presence) and the dependent variables (satisfaction and reported 
learning), it shows that the central construct of teaching presence does 
have a significant effect on learners. 

The Effects of Teaching Presence on Critical Thinking 

Dewey (1933) believed that the main benefit of critical thinking 
was that it helps the learner develop and deepen the picture of their 
experiences. For this reason, critical thinking is important at every stage 
of learning. Critical thinking allows the learner to assess the quality of 
their current knowledge and incoming knowledge. It also allows the 
learners to develop knowledge of their own. As Dewey notes, learners 
and the communities they hail from cannot be meaningfully separated. 
According to Garrison and Archer (2000), the advantage of Dewey’s 
framework of reflective thinking is that most forms of active cognition 
(critical, abstract, or inference for example) can be explained by the 
theory. The core principle of the model, which moves through 
imagination, deliberation and action, can be li nk ed easily to most 
learners’ experiences of e-leaming. This is even more the case when the 
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e-leaming is asynchronous and written. This creates a discourse that is 
heavily weighted towards reflection, as opposed to most verbal 
discourses that tend to be spontaneous and lack reflection (Garrison et al. 
2000). Research involving the analysis of discourse has shown the 
relationship between features of online discourse that make up critical 
thinking and learning (Pilkington, 2001). Pilkington’s analysis allows us 
to indirectly understand how the process of discussion intersects with 
processes inside learner’s minds. Within the community of inquiry, 
critical thinking can be considered to be synonymous with with cognitive 
presence (Garrison & Archer, 2000; Garrison & Akyol, 2013). Other 
authors have suggested that cognitive presence facilitates or occurs in 
conjunction with critical thinking (Gasevic, Adesope, Joksimovic & 
Kovanovic, 2015; Lee, 2013). Furthermore, the Col along with cognitive 
presence has been modeled together conceptually with critical thinking as 
an effective way to understand student behavior (Goh, Dexter, & Self, 
2014). 


In Prasad’s (2009) study of the effects of teaching presence on 
critical thinking, he found that levels of teaching presence were positively 
correlated with levels of critical thinking. This clearly connects with 
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Dewey’s (1933) work, where he discussed the idea that the development 
of higher order critical thinking skills “ appeared in student discussions 
only when prompted by specific instructional techniques” (pg. 9). He 
claimed that collaborative solutions tended to be introduced when the 
teacher prompted the learners to move towards those kinds of solutions. 

In Dringus, Snyder and Terral’s (2010) examination of the effects 
of facilitating discourse on student participation, satisfaction and content 
understanding, they measured learner’s perceptions of the instructors’ 
audio contributions. They found that the learners felt more connected to 
the course and that learners felt that facilitation managed to improve their 
understanding of the content. A further study by Sheridan and Kelly 
(2010), investigating which parts of teaching presence students felt were 
valuable, found that responsiveness and clarity were the features that 
were most valued by learners. Shea, Hays, Uzuner-Smith, Gozza-Cohen, 
Vickers, and Bidjerano (2014) have further established the relationships 
between teaching presence and cognitive presence and that relationship 
with learning. In research by Kim (2015), she found that enhancing 
scaffolding resulted in higher levels of critical thinking and more even 
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participation among study participants when compared to minimal 
scaffolding. 

METHODS 

Subjects and Context 

The 219 participants for this experiment were taking English 
classes focused on preparing them for the Korean teachers entrance exam 
(im-yong-gyo-shi) over three semesters in 2013 and 2014. Permission 
was given by the college of education to experiment on the students 
involved in this study. Furthermore, participants filled out a consent fonn 
giving their consent for their posts to be used for research purposes (see 
appendix 1). This study takes the posts generated by the users of an 
online forum as part of a blended learning environment, with the online 
posting meant to support and further develop the students’ offline 
discourse and writing skills in the hope that this will develop their ability 
to generate meaningful understand of issues pertaining to class 
management and delivering instruction. Offline course activities included 
lectures, group work and presentations. The main online component of 
the course was the students’ use of an asynchronous message board 
where they could post their ideas and respond to others’ ideas related to 
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the course materials. The gender and major breakdown for the classes can 
be seen in Table 1. 

Table 1 

The Gender and Majors for the Three Forums 



Semester 

one 

Semester 

two 

Semester 

Three 

Total 

Gender 

Male 

24 

26 

27 

77 

Female 

46 

46 

50 

142 

Major 

English 

37 

35 

40 

112 

Special 

6 

6 

2 

14 

Business 

1 

2 

1 

4 

Pedagogy 

3 

1 

2 

6 

Art 

2 

2 

4 

8 

Life Skills 

5 

5 

5 

15 

Ethics 

1 

3 

2 

6 

Early Childhood 

2 

2 

2 

6 

Literature* 

1 

2 

2 

5 

Social Studies 

2 

2 

5 

9 
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Calligraphy 

0 

2 

0 

2 

Korean 

2 

2 

3 

7 

Music 

0 

0 

2 

2 

Tourism* 

0 

1 

0 

1 

Chemistry 

5 

2 

2 

9 

History 

2 

1 

1 

4 

Earth Science 

1 

1 

3 

5 

Economics* 

0 

1 

0 

2 

Geography 

0 

1 

1 

2 

Total 

70 

72 

77 

219 


All majors were part of the college of education except those marked 
with an * 

Experimental Procedures 


This study was conducted over the course of a year and a half (3 
semesters) and involved varying the nature of the learning environment 
in which learners interacted. To formulate the instructional design aspects 
of the experiment, a modification of the Instructional Design for Online 
Learning model (IDOL) designed by Siragusa, Dixon and Dixon (2007) 
was used. Again, three experimental conditions were created with 
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progressively higher levels of instructor control over the learning 
environment. 

Defining the Learning Environments 

A modified version of the Instructional Design for Online 
Learning model (IDOL) designed by Siragusa, Dixon and Dixon (2007) 
to understand the variations in the learning environments. This paper 
takes seven instructional design decisions for three different learning 
communities (online forums) and shows their variation (fig. 2). 

Content source is the amount to which students or instructors 
generate the content. In the first forum, the content was totally free. 
Participants in the forum were able to post on whatever topics they 
wished. This contrasts with the second and third forums, which had 
specific instructor generated topics. 

Linear content is the degree to which the content follows a set path in 
terms of narrative, difficultly or some other criteria. In the first and 
second forum, the content followed no clear linear pattern, while in the 
third, there was a clearly mapped out path from more difficult questions 
to simpler ones. 

Instructor posting is the degree to which the instructor interacts and 
guides the interactions and discourse in the learning environment. On the 
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first two forums, there were few posts made by any instructors. However, 
on the third forum, an instructor is actively involved in controlling 
interactions and moderating the discourse. 

Structure and organization refers to the amount that the learners’ 
discourse and interactions are structured and controlled. This could be in 
the fonn of being given examples to follow or word limits. In the first 
forum, there was no structure. Learners were not given examples and 
were told that any style of writing was appropriate. In the second forum, 
specific questions were given to students and the types of responses they 
were asked to give were clearly defined (examples were given of strong 
and weak posts). Further to this, in the third forum, participants were 
restricted in that their replies must make clear contributions to the topic 
being discussed and posts should focus on the issues brought up by other 
posters. 

Study flexibility (when, where) is how free the learners are to 
contribute in regards to time and space. On all three forums, there were 
no spatial limitations (the learning environment supported all OSs). On 
the first forum, timing was not an issue; students would be assessed based 
on their total production at the end of the course. This differs from the 
second forum and the third forum on which students were assessed 
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monthly. Furthermore, on both the second and third forum, learners were 
encouraged throughout the experiment to continue to post. 

Interaction is the amount of instructor control over learner 
interaction. On the first forum students were able to interact with 
whomever they liked, moreover students were told that they didn’t have 
to interact if they chose not to. In the second forum, the learners could 
interact with whomever they wanted, however they were told that 
interaction was compulsory. These can be contrasted with the third 
forum, in which learners were put into 6 equally sized sub-forums and 
asked to interact only with learners in their group. 

Feedback is the number of times that their online contributions were 
commented on by their instructor. On the first forum, this was not done 
systematically (at the learner’s request). On the second forum and on the 
third forum, feedback was provided every two weeks. 

The purpose of Figure 2 is to visually show the differences in levels 
of instructional design based on the seven proceeding elements. If the 
forums are tagged to the left of the spectrum, then one can think of them 
as being freer and more learner controlled. When the forums are tagged 
to the right, it indicates that the forum is more instructor controlled in 
regards to that particular indicator. The points at which each forum sits 
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on the indicator is subjective. The figure’s purpose is to give a visual 
representation of qualitative differences among the three forums. 



□ 

O 



: Forum 1 
: Forum 2 

: Forum 3 


Figure 2. Modeling the three forums in terms of instruction design features 

Coding Critical Thinking and Social Presence 

In the learning environments studied in this research, there were 
more than nine thousand posts and nearly two thousand threads. It was 
necessary to reduce the data set to be coded. For this reason, 900 posts 
were randomly selected to be analyzed for levels of critical thinking and 
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social presence. The sample was generated by randomly selecting a 
thread within the forum, then randomly selecting a post in that thread. 
Sampling was done at the level of the thread first because, while analysis 
was going to be conducted at the level of the post, for each individual 
post’s context (the posts around it) would be required for the coder to 
make correct coding choices. The procedure for coding the posts follows 
a structure laid out more clearly in Costley and Han (2013) in which the 
data goes through an eight step process where by the 1) sample is chosen, 
2) the unit size decided, 3) coding scheme is implemented, 4) the method 
of implementing the coding scheme is chosen, 5) representing the data in 
a fonn it can be analyzed, 6) analyzing the data, 7) interpreting the 
analysis, and 8) repeating the process for clarity. 

The coding scheme used for measuring critical thinking in this 
paper is based on Newman, Webb, and Cochrane’s (1996) model for 
assessing the levels of critical thinking in online environments. Newman 
et al. (1996) describe 10 categories of critical thinking: relevance, 
importance, novelty, outside knowledge, ambiguities, linking ideas, 
justification, critical assessment, practical utility, and width of 
understanding (see appendix 2). Each of the varied codes can be 
designated by a (+) or a (-) symbol. This represents whether or not the 
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statement enriches (+) or detracts from (-) the creation of a discourse rich 
in critical thinking. However, this paper will use a modification of the 
above critical-thinking coding scheme. The reason for this is the ratio 
between 1 and -1 created by Newman, Webb, Cochrane’s (1996) coding 
system is difficult to connect with the outputs of Rourke et al.’s (1999) 
coding scheme, which is a scale from 0 to 9. For this reason, this research 
breaks from Newman, Webb and Cochrane’s method in that the resulting 
data would be in the form of a number of events to represent critical 
thinking as opposed to a ratio. This was done so that it was more in line 
with the coding scheme used to analyze social presence, which made the 
resulting analysis easier and more useful. 

Social presence was measured using the coding scheme from 
Rourke et al.’s (1999) Assessing Social Presence in Asynchronous Text- 
Based Computer Conferencing. This paper lays out three categories, 
which form the basis of teaching presence: affective, interactive, and 
cohesive. There are nine indicators from that coding scheme used in this 
research to generate the social presence score: (see Appendix 3 for the 
full coding scheme and examples). 


Two raters, the author and one other university professor, 
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experienced with online learning coded the data. Initially, the indicators 
for each construct were discussed, and then ten posts were coded together 
with discussion about each post. Once that initial stage of discussion was 
complete, 90 posts were coded by both raters to examine the level of 
inter-rater reliability. The resulting Cohen’s kappas were 0.91 for social 
presence and 0.86 for critical thinking. Both of these values were 
considered acceptable and the full set of 900 posts were split in half and 
coded. After the initial coding, to further check the reliability of the 
coding scheme, two more coders checked the already coded samples. 

This meant that each post was initially coded, and each post was 
subsequently reread, with the codes included, by two more separate 
coders. There was a high degree of rater agreement from the checked 
codes with a Cohen’s kappa of 0.96 for social presence and 0.92 for 
critical thinking. Internal reliability was also measured with the social 
presence construct having a Cronbach’s alpha of .78. The internal 
reliability of the critical thinking construct was slightly lower with a 
Cronbach’s alpha value of .75. Both of these values are considered 
acceptable in research of this kind (Streiner, 2003) and the constructs of 
critical thinking and social presence was considered reliable enough for 
analysis. 
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RESULTS 

The overall levels of social presence and critical thinking were 
dissimilar in that the average level of critical thinking per post was 3.66, 
while the average level of social presence per post was 1.99 (see table 2). 
Table 2 


Mean Levels of Critical Thinking and Social Presence Per Post 



N 

Minimum 

Maximum 

Mean 

Std. 

Deviation 

Social 

900 

0 

9 

1.99 

2.168 

presence 






Critical 

900 

0 

9 

3.66 

2.134 

thinking 







To answer the two research questions asked in this study, a 
comparison of means was used. In terms of social presence, as can be 
seen in table 3, the learning environment with the highest level of social 
presence was the low control learning environment (2.95 social presence 
average per post), then the medium control (1.99), and finally the high 
control learning environment (1.04). 





The Journal of Educators Online-JEO January 2016 ISSN 1547-500X 

Vol 13 Number 1 

140 

Table 3 


Mean Social Presence for the Three Conditions 



N 

Mean 

SD 

Low control 

300 

2.95 

2.64 

Medium control 

300 

1.99 

1.94 

High control 

300 

1.04 

1.25 


When examining the means for critical thinking, the ranking was 
reversed, with the highest level of critical thinking occurring in the high 
control learning environment (4.54 average critical thinking per post), 
then the medium control learning environment (3.83), and finally the low 
control learning environment (2.06). 


Table 4 


Mean Critical Thinking for the Three Conditions 



N 

Mean 

SD 

Low control 

300 

2.06 

1.72 

Medium control 

300 

3.83 

1.96 

High control 

300 

4.54 

2.22 








The Journal of Educators Online-JEO January 2016 ISSN 1547-500X 

Vol 13 Number 1 

141 

As can be seen from the previous two tables, there is a clear 
difference between the levels of critical thinking and the levels of social 
presence depending on which learning environment they were in. To 
further examine whether these differences are statistically significant, 
ANOVA was used. As Table 5 shows, when comparing average levels of 
social presence between all three learning environments, there is a 
statistically significant difference. 

Table 5 


ANOVA (Scheffe test) for Comparing Social Presence Means 



Low 

Medium 

High 


control 

control 

control 

Low control 

0 

0.96* 

1.91* 

Medium control 

- 0.96* 

0 

0.95* 

High control 

- 1.91* 

-0.95* 

0 


*. The mean difference is significant at the 0.05 level 


The same comparison was done with the average levels of critical 
thinking per post in each learning environment. Table 6 shows that, as 
with social presence, each learning environment has a statistically 
significant mean difference from the other learning environments. 
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Table 6 


ANOVA (Scheffe test) for Comparing Critical Thinking Means 



Low 

Medium 

High 


control 

control 

control 

Low control 

0 

- 1.23* 

-1.94* 

Medium control 

1.23* 

0 

-0.71* 

High control 

1.94* 

0.71* 

0 


*. The mean difference is significant at the 0.05 level 


To further examine the effects from the experimental conditions, 
the individual indicators that make up social presence and critical thinking 
are displayed in Tables 7 and 8. As can be seen in Table 7, the most 
common indicator of social presence in all three learning environments 
was not consistent. Within the low and medium control environments the 
most common indicator was expressing emotions, while in the high control 
environment the most common indicator was expressing agreement. 
Looking at Table 7, this seems to be caused by expressing agreement not 
lowering as much as the other indicators across the 3 learning 
environments. Regardless of that, every indicator was lower in the medium 
control than the low control, and lower still in the high control 


environment. 
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Table 7 

Individual Indicators that Generate Social Presence 



Low control 

Medium 

control 

High 

control 

Total 

Expressing 

Agreement 

136 

122 

117 

375 

Expressing 

Emotion 

160 

139 

71 

370 

Complimentin 

g 

134 

79 

31 

244 

Self disclosure 

120 

72 

27 

219 

Asking 

questions 

87 

54 

15 

156 

Phatics 

84 

36 

21 

141 

Referring to 

others’ 

messages 

65 

51 

13 

129 





The Journal of Educators Online-JEO January 2016 ISSN 1547-500X 

Vol 13 Number 1 

144 

Using 46 22 14 82 

inclusive 

pronouns 

Using humor 52 22 4 78 


As with social presence, the individual indicators that are used to 
generate the critical thinking score were examined (see table 8). 
Relevance was the most common indicator in all three learning 
environments, though in the high control environment there were 268 
cases of both relevance and importance. For most indicators, there were 
more cases of each particular code in the medium control environment 
than in the low control environment, and in the high control environment 
than in the medium control environment. However, there were some 
indicators that didn’t follow that trend. The indicator linking ideas was 
more common in the low control environment than the medium control 
environment. This showed that the students were more likely to link ideas 
together when they were given a more structured environment, and that 
the less controlled environment had fewer ideas that were directly linked 
together. Furthermore, two indicators, bringing in new information and 
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critical assessment were more common in the medium control 
environment than the high control environment. 

Table 8 


Individual Indicators that Generate Critical Thinking 



Low 

control 

Medium 

control 

High 

control 

Total 

Relevance 

243 

253 

268 

764 

Importance 

214 

250 

268 

732 

Linking ideas 

110 

96 

195 

401 

Practical utility 

85 

98 

166 

349 

New information 

66 

153 

126 

345 

Justification 

75 

94 

170 

339 

Outside 

33 

102 

132 

267 

knowledge 





Critical 

26 

90 

69 

185 

assessment 





Width of 

16 

66 

76 

158 

understanding 





Clearing up 

22 

42 

87 

151 

ambiguities 
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DISCUSSION 

The Effect of Design Control Level on Critical Thinking 

The learning environments differed significantly from one another 
in terms of critical thinking. The first forum, which had the least 
instructor focused design features, had the lowest level of critical 
thinking. This was followed by the second forum, which had a moderate 
(for this experiment) level of critical thinking. The third forum, which 
had the highest level of instructor focus among the differing forums 
examined in this study, had the highest level of critical thinking. These 
differences where statistically significant between all three forums. 

The positive effect that instructor control has on critical thinking 
meshes well with other research on this topic, which has shown that when 
students attempt to broaden their ideas and make judgments, direct 
instruction is more effective than indirect discovery learning (Klahr and 
Nigam, 2004). Fisher (2001) has shown that some students develop levels 
of critical thinking through general educational processes. However, 
supplemental instruction in the form of examples of abstract reasoning 
skills from instructors increases students’ ability to process information 
critically. Ke (2010) also showed that direct instruction is the most 
important feature when predicting reported knowledge construction. 
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Furthermore, Ke (2010) demonstrated that clear written assignments are 
found to contribute more than any other task to learners’ construction of 
knowledge. In this study, as the environment became more controlled, 
templates were provided and became progressively more detailed and 
emphasized, which could allow learners the clarity to write in a way more 
conducive to critical thinking. 

Akyol and Garrison’s (2008) paper showed that learners 
responded to collaborative regular tasks with higher levels of cognition. 
They claimed that when students were required to engage together on a 
schedule, they were more likely to develop complex and well thought out 
solutions to problems. Diaz, Swan, Ice, and Kupczynksi (2010) have 
shown that asynchronous discussions that gave students the opportunity 
to work together led to higher levels of cognitive presence when their 
instructor had created an environment for them to interact effectively in. 
Both of these studies support the idea of collaboration being a gateway to 
developing cogitative presence or critical thinking. The fact that the low 
control environment had no collaboration required, the medium control 
environment had forced collaboration and interaction, and the high 


control environment had forced collaboration and interaction within set 
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groups, may explain the increase in critical thinking as the learning 
environments became more controlled. 

The Effect of Design Control Level on Social Presence 

There were clear differences among the learning environments in 
regards to social presence as well. Forum 3 had the lowest level of social 
presence, forum 2 had a moderate amount (for this experiment) and 
forum 1 had the highest level of social presence among the three different 
learning environments. These differences in regards to the levels of social 
presence were statistically significant. It seems clear that more instructor 
focused levels of instructional design positively affect critical thinking 
and negatively affect social presence. 

As mentioned above, Akyol and Garrison (2008) and Diaz, Swan, 
Ice, and Kupczynksi (2010) have shown the positive effects that more 
control may have on critical thinking. However, they both mention that 
forced regular interaction may at best be neutral or at worst negative for 
social presence among learners. Shea and Bidjerano (2009) also 
commented that when tasks were clearly directed in online learning, 
social presence effects could be non-existent. Shea and Bidjerano (2009) 
also noted that learners sometimes experienced low social presence in 
conjunction with high teaching presence, though their teaching presence 
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measures were not clearly delineated between differing instructor 
presence types. 

The results from this research break with Aragon’s (2003) work 
describing the methods by which an instructor can develop and maintain 
social presence in an online environment. He claimed that a wide variety 
of instructor behaviors would induce higher levels of social presence. 
These would include both instructor posts that would fall into the 
categories of direct instruction and facilitating discourse used in this 
experiment. While this explains the benefits from facilitating discourse 
found in this experiment, it does not explain the lack of effect direct 
instruction has when compared to no instructor posting. 

CONCLUSION 

This research has shown that the amount of social presence will 
decrease as learning environments become more instructor controlled, 
and that learners will have a more critical discourse if the learning 
environment is more controlled. The lowering of the amount of social 
presence as the environments became more teacher-controlled could be a 
concern. As is the lack of critical thinking when the environments are 
free from instructor control. The variation in social presence and critical 
thinking caused by instructor intervention also has some tantalizing 
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implications. If an instructor wishes to maximize critical thinking he/she 
may only do so to the detriment of social presence. On the flip side, less 
control can increase levels of social presence. The fact that those learning 
environments have lower levels of critical thinking may be a concern for 
instructors intending to manipulate student discourse in one direction or 
another. There is an implication that unlike previous research, there may 
be no relationship or a negative relationship between critical thinking and 
social presence. This has been seen in criticisms of the Col in other 
research, which suggest that the effect of social presence on cognitive 
presence (and vice versa) has been exaggerated and that other factors 
related to the learning environment and the students themselves are more 
important (Annand, 2011; Nagel & Kotze, 2011). 

This leads to the conclusion that the goals of any learning 
environment must be taken into consideration when designing a forum 
for learners to use. Modification can lead to changes in outputs; therefore 
instructors must focus on what design decisions will best suit the students 
under their charge. The tension between social presence and critical 
thinking further demonstrates the importance of care when setting up and 
delivering online instruction. Consideration must be given to the idea that 
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higher levels of interaction are favorable to lower levels (Tan, Tripathi, 
Zuiker and Seah, 2010). 

These findings mimic the general findings in academia regarding 
the importance of instructor intervention in e-learning environments. 
When asked, learners respond that instructor involvement is crucial to 
academic success and engagement (Hughes & Daykin, 2002; Rourke & 
Anderson, 2002; Salmon 2002; Shea, 2003). This shows the importance 
of design choices when developing online learning environments for 
students to interact on. 

Limitations and Further Research 

This study’s subjects represent a subsection of learners using 
online environments, which limits the ability to generalize from the 
results of this study. Furthermore, the 3 differing learning environments 
have been spread over 3 different semesters. This means that there may 
be some differences between the groups, even though they are drawn 
from the same population and have similar profiles in terms of age, 
gender, and grades. For this reason, the results of this study are not 
generalizable to all leamer-to-leamer interactions online, but provide a 
basis for further study into the effects of differing learning environments 


and learner discourse. 
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Of great interest and in need of further study is the question 
whether there is a more intimate relationship between levels of critical 
thinking and social presence. It may be that critical thinking and social 
presence levels may be in direct tension with one another, which will 
cause a quandary when designing online learning tasks. This research and 
the tensions it describes must be considered when designing online 
learning environments. It is the intention of this author to take the 
framework proposed here and the data analyzed in this study and subject 
it to other more developed forms of analysis. 
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Appendix 1. 

Research Participant Information and Consent Form and 
Demographic Data 

1. EXPLANATION OF THE RESEARCH 

You will be involved in a study investigating the effects of teaching in an 
online environment. The posts you will produce, as part of this class will 
be analyzed to understand how different types of learning environments 
affect how and what you post. Your information will remain confidential 
and you will never be identified. 

2. YOUR RIGHTS TO PARTICIPATE, SAY NO, OR WITHDRAW 

Participation in this research project is completely voluntary. You have 
the right to say no. You may change your mind at any time and withdraw. 
Whether you choose to participate or not will have no effect on your 
grade or evaluation. 

3. CONTACT INFORMATION FOR QUESTIONS AND 
CONCERNS: 

If you have concerns or questions about this study, such as scientific issues, 
how to do any part of it, or to report an injury or a great deal of stress, 
please contact the researcher: 

Jamie Costley, 010 2974 6014, costley@kongiu.ac.kr 
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4. DOCUMENTATION OF INFORMED CONSENT. 

Your signature below means that you voluntarily agree to participate in 
this research study. 

Name (in Korean):_ 

Sign:_ 

Date:_ 

Research Questionnaire (Demographics) 

1) What is your gender?_ 

2) What is your age?_ 

3) What year are you in at your university?_ 

4) What is your Major?_ 

What is your online username?_ 

Appendix 2 

Newman, Webb and Cochrane’s (1996) coding schema 

Category Positive Indicator 

R+ Relevance R+ Relevant statements 

1+ Importance 1+ Important points/issues 

N+- Novelty. New info, ideas, NP+ New problem-related information 
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Solutions NI+ New ideas for discussion 

NS+ New solutions to problems 
NQ+ Welcoming new ideas 


NL+ learner (student) brings 
new things in 

0+ Bringing outside 

OE+ Drawing on personal experience 

knowledge or 

experience to bear on problem OC+ Refer to course material 

OM+ Use relevant outside material 
OK+ Evidence of using previous 
Knowledge 

OP+ Course related problems brought in 
OQ+ Welcoming outside knowledge 


A+ Ambiguities: clarified or 
confused 


AC+ Clear, unambiguous statements 


L+ Linking 
interpretation 


A+ Discuss ambiguities to clear them up 

ideas, 

L+ Linking facts, ideas and notions 
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J+ Justification 


C+ Critical assessment 

P+ Practical 
(grounding) 


L+ Generating new data from 
information collected 


JP+ Providing proof or examples 

JS+ Justifying solutions or judgments 

JS+ Setting out advantages and 
disadvantages of situation or solution 

C+ Critical assessment/evaluation of own 
or others' contributions. 

CT+ Tutor prompts for critical evaluation 

utility 

P+ relate possible solutions to 

familiar situations 


P+ discuss practical utility of new ideas 
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W+ Width of understanding W+ Widen discussion (problem within a 
(complete picture) larger perspective. Intervention strategies 

within a wider framework.) 


Appendix 3 

Indicators of Social Presence 


Category 


Affective 


Indicators 


Expressio 
n of 

emotions 


Use of 
humor 


Definition 

Conventional expressions 
of emotion, or 
unconventional 
expressions of emotion, 
includes repetitious 
punctuation, conspicuous 
capitalization, emoticons. 
Teasing, cajoling, irony, 
understatements, 
sarcasm. 


Example 


I just can't stand it 
when ...!!!!" 

"ANYBODY OUT 
THERE!" 


The banana crop in 
Edmonton is looking 
good this year) 
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"Where I work, this 


Self¬ 

disclosure 

Presents details of life 

outside of class, or 

expresses vulnerability. 

is what we do ..." "I 

just don’t understand 

this question" 


Using reply feature, 

Software dependent, 

Interactiv Continuin 


rather than starting a new 

e.g., "Subject: Re" or 

e g a thread 


thread. 

"Branch from" 

Quoting 

Using software features 

Software dependent, 

from 

to quote others entire 

e.g., "Martha writes:" 

others' 

message or cutting and 

or text prefaced by 

messages 

pasting selections of 

less-than symbol <. 

Referring 

others' messages. 

"In your message, 

explicitly 

Direct references to 

you talked about 

to others' 

contents of others' posts. 

Moore's distinction 

messages 


between..." 


Students ask questions of 

"Anyone else had 

Asking 


other students or the 

experience with 

questions 


moderator. 


WEBCT? 
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Cohesive 


Complime 



nting, 

Complimenting others or 

"I really like your 

expressing 

contents of others' 

interpretation of the 

appreciati 

messages. 

reading" 

on 




Expressin 

g 

agreement 


Vocatives 


Expressing agreement 
with others or content of 
others’ messages. 

Refering to group 
members by name 


"I was thinking the 
same thing. You 
really hit the nail on 
the head." 

"I think John made a 
good point." 


Addresses 
or refers to 
the group 
using 
inclusive 
pronouns 
Phatics, 
salutations 


Addresses the group as, 
"us, we, our". 


"Our textbook refers 
to...""I think we 
veered off track..." 


Communication that "Hi all" "Thaf s it for 
serves a purely social now" "We're having 
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function; greetings, the most beautiful 
closures. weather here" 

Rourke et al. (1999) pg. 61 



